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Abstract 

We consider rate R = - causal linear codes that map a sequence of fc-dimensional binary vectors 

<N 

^ {bt}'^(, to a sequence of n-dimensional binary vectors {ct}'^(,, such that each Ct is a function of {6,-}* ^g- 

^ ^ Such a code is called anytime rehable, for a particular binary-input memoryless channel, if at each time 

instant t, and for all delays d > do, the probability of error P (bt-d\t h-dj decays exponentially in 
^ d, i.e., P (bt-d\t bt-d) < ri2~^'^'^, for some /3 > 0. Anytime reliable codes are useful in interactive 

I I communication problems and, in particular, can be used to stabilize unstable plants across noisy channels. 

^ Schulman proved the existence of such codes which, due to their structure, he called tree codes in [1]; 

however, to date, no exphcit constructions and tractable decoding algorithms have been devised. In this 
paper, we show the existence of anytime rehable "hnear" codes with "high probability", i.e., suitably 
chosen random hnear causal codes are anytime reliable with high probabihty. The key is to consider 

<N 

l/-) time-invariant codes (i.e., ones with Toeplitz generator and parity check matrices) which obviates the 

cn 

need to union bound over all times. For the binary erasure channel we give a simple ML decoding 
algorithm whose average complexity is constant per time iteration and for which the probabihty that 
complexity at a given time t exceeds KC^ decays exponentially in C. We show the efficacy of the 
J> method by simulating the stabilization of an unstable plant across a BEC, and remark on the tradeoffs 

between the utihzation of the conmiunication resources and the control performance. 

I. Introduction 

Shannon's information theory, in large part, is concerned with one-way communication of a message, 
that is available in its entirety, over a noisy communication network. There are increasingly many 
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applications such as networked control systems [2] and distributed computing [1] where communication is 
not one-way but interactive. A networked control system is characterized by the measurement unit begin 
separated from the control unit by a communication channel. The interactive nature of communication can 
be understood by observing that the measurements to be encoded are determined by the control inputs, 
which in turn, are determined by the encoded measurements received by the controller. Block encoding 
of the measurements is not apphcable anymore because the controller needs real time information about 
the system so that an appropriate control input can be apphed. This is especially critical when the system 
being controlled is open loop unstable. Any encoding-decoding delay translates into the system growing 
increasingly unstable. Hence, the desired reliability of communication is determined by the quahty of 
the control input needed to stabihze the system. 

In the context of rate-hmited deterministic channels, significant progress has been made (see eg., [3], 
[4]) in understanding the bandwidth requirements for stabilizing open loop unstable systems. When the 
communication channel is stochastic, [5] provides a necessary and sufficient condition on the communica- 
tion reliability needed over a channel that is in the feedback loop of an unstable scalar linear process, and 
proposes the notion of anytime capacity as the appropriate figure of merit for such channels. In essense, 
the encoder is causal and the probability of error in decoding a source symbol that was transmitted d 
time instants ago should decay exponentially in the decoding delay d. 

Although the connection between communication reliability and control is clear, very little is known 
about error-correcting codes that can achieve such reliabihties. Prior to the work of [5], and in a different 
context, [1] proved the existence of codes which under maximum likelihood decoding achieve such 
reliabilities and referred to them as tree codes. Note that any real-time error correcting code is causal 
and since it encodes the entire trajectory of a process, it has a natural tree structure to it. [1] proves the 
existence of nonlinear tree codes and gives no explicit constructions and/or efficient decoding algorithms. 
Much more recently [6] proposed efficient error correcting codes for unstable systems where the state 
grows only polynomially large with time. So, for hnear unstable systems that have an exponential growth 
rate, all that is known in the way of error correction is the existence of tree codes which are, in general, 
non-linear. When the state of an unstable scalar linear process is available at the encoder, [7] and 
[8] develop encoding-decoding schemes that can stabilize such a process over the binary symmetric 
channel and the binary erasure channel respectively. But when the state is available only through noisy 
measurements, little is known in the way of stabihzing an unstable scalar linear process over a stochastic 
communication channel. 

The subject of error correcting codes for control is in its relative infancy, much as the subject of block 



coding was after Shannon's seminal work in [9]. So, a first step towards realizing practical encoder- 
decoder pairs with anytime reliabilities is to explore Unear encoding schemes. We consider rate R = ^ 
causal linear codes which map a sequence of A;-dimensional binary vectors {6t-}?^o ^ sequence of 
n— dimensional binary vectors {ct-}^q where q is only a function of {6t-}*^o- Such a code is anytime 
reliable if at all times t and delays d > do, P{bt-d\t / h-d) < j^2~'^"'^ for some /3 > 0. We show that 
Hnear tree codes exist and further, that they exist with a high probability. For the binary erasure channel, 
we propose a maximum likelihood decoder whose average complexity of decoding is constant per each 
time iteration and for which the probability that the complexity at a given time t exceeds KC^ decays 
exponentially in C. This allows one to stabilize a partially observed unstable scalar linear process over 
a binary erasure channel and to the best of the authors' knowledge, this has not been done before. 

In Section n, we present some background and motivate the need for anytime reliability with a simple 
example. In Section HI, we come up with a sufficient condition for anytime reliability in terms of the 
weight distribution of the code. In Section IV, we introduce the ensemble of time invariant codes and use 
the results from Section III to prove that time invariant codes with anytime reliability exist with a high 
probability. In Section V, we present a simple decoding algorithm for the BEC and present simulations 
in Section VI to demonstrate the efficacy of the algorithm. 

II. Background and Problem Setup 

Owing to the duality between estimation and control, the essential complexity of stabiUzing an unstable 
process over a noisy communication channel can be captured by studying the open loop estimation of the 
same process. So, we will illustrate the kind of communication reliability needed for control by analyzing 
the open loop estimation of the following random walk. 

A toy example: Consider estimating the following random walk, xt+i = Xxt + wt, where wt = ±1 w.p 
2, xo = and |A| > 1. Suppose an observer observes xt and communicates over a noisy communication 
channel to an estimator. The observer clearly needs to communicate one bit^ telling whether wt is +1 
or —1. Now the estimator's estimate of the state, x^^i^^, is given by 

t 

Xt+l\t = ^t-jWj\t (1) 

i=o 

Suppose = P (argminj(it;j|^ ^ Wj) = t — d+l), i.e., the position of the earliest erroneous Wj\t is 



'if the channel is binary input, it should clearly allow at least one usage for each time step of the system evolution 



at time j = t — d + 1. From (1), we can write — as 

n 2 t 

J2 p{wo:t,wo.,t\t) J2^*~^('^j-'^j\t) =J2^lt J2 

«'o:t,'i'>0:t|t j=l d<t j=t-d+l 



< 



(|A| 



d<t 



Clearly, a sufficient condition for limsup^ E \ xt+i — | to be finite is as follows 

Pd,t < |A|-(2+'^)'^ V d > do, i > and (5 > (2) 

where do and to are constants that do no depend on d. In the above example, noise is discrete valued. 
If it is not discrete, e.g., a uniform random variable on a bounded interval, then apart from the reliabihty 
criterion above, one would also need a minimum rate of communication that depends on the size of 
log2 |A|. But codes that communicate at a positive rate and satisfy the above reliability criterion have 
been more or less elusive. Consider the following unstable scalar hnear process 
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Fig. 1: System model 



xt+i = Xxt + Ut + wt, yt = xt + vt (3) 

where |A| > 1, ut is the control input and, \wt\ < \ and \vt\ < are bounded process and measurement 
noise variables. The measurements {yt} are made by an observer while the control inputs {ut} are applied 
by a remote controller that is connected to the observer by a noisy communication channel. Naturally, 
the measurements yo:t-i will need to be encoded by the observer to provide protection from the noisy 
channel while the controller will need to decode the channel output to estimate the state xt and apply 
a suitable control input ut. This can be accomplished by employing a channel encoder at the observer 
and a decoder at the controller. For simplicity, we will assume that the channel input alphabet is binary. 
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Fig. 2: Causal encoding and decoding 



Suppose one time step of system evolution in (3) corresponds to n channel uses^. Then, at each instant 
of time t, the operations performed by the observer, the channel encoder, the channel decoder and the 
controller can be described as follows. The observer generates a A;— bit message, bt € {0, l}'^, that is a 
causal function of the measurements, i.e., it depends only on yo:t- Then the channel encoder causally 
encodes 6o:i G {0, 1}'^* to generate the n channel inputs q G {0, 1}". Note that the rate of the channel 
encoder is R = k/n. Denote the n channel outputs corresponding to a by zt G Z"-, where Z denotes the 
channel output alphabet. Using the channel outputs received so far, i.e., ZQ-t G the channel decoder 
generates estimates {b^\t}T<t of {br}T<t, which, in turn, the controller uses to generate the control input 
ut+i- This is illustrated in Fig. 2. Note that we do not assume any channel feedback. Then using the 
lattice quantizer argument presented in [5], in the limit of large n and large A, we have the following 
sufficient condition on the performance of the encoder-decoder pair so that the unstable process in (3) 
can be stabilized 

Lemma 2.1 (Theorem 5.2 [5]): It is possible to control the unstable scalar process (3) over a noisy 
communication channel so that limsup^Elxfl'" < oo if, for some rate R > ilog2|A| and exponent 
/3 > ^ log2 |A|, we have 

P (mm{T : b^^ ^br} = t-d+l^ < t]2-^'^'^, V d > do, * > to 

where do and to are constants independent of d, t. 

In what follows, we will demonstrate causal linear codes which under maximum likelihood decoding 
achieve such exponential reliabilities. 

^In practice, the system evolution in (3) is obtained by discretizing a continuous time differential equation. So, the interval 
of discretization could be adjusted to correspond to an integer number of channel uses, provided the channel use instances are 
close enough. 



III. Linear Anytime Codes - A SuFnciENT Condition 

As discussed earlier, a first step towards developing practical encoding and decoding schemes for 
automatic control is to study the existence of linear codes with anytime reliability. We will begin by 
defining a causal linear code. 

Definition 1 (Causal Linear Code): A causal linear code is a sequence of linear maps : {0, 1}'^'^ 
{0, 1}" and hence can be represented as 



Grlh + Gr2b2 + ■■■ + Grrbj 



(4) 



1 nxfc 



where e {0, 1}' 

We denote Cr = /r(^i:r)- Note that a tree code is a more general construction where need not be 
linear. Also note that the associated code rate is i? = ^ . One can alternately represent a causal linear code 
by an infinite dimensional block lower triangular generator matrix Gn,R or equivalently as an infinite 
dimensional block lower triangular parity check matrix, !!„ i^. 



Gn ... 
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(5) 



where Hij e {0, l}"^" and n = n(l — RY. In fact, we present all our results in terms of the parity 
check matrix. Before proceeding further, it is useful to introduce some notation 



A. Notation 

1) j^ = nt X nt leading principal minor of M.n,R 

2) Ct^{ce{0,ir':ml^c = 0} 

3) = {ceCt: Cr<t-d+i = 0, ct-d+i 7^ 0} 

4) K,a=\{c^Ct,d--\\c\\=w}\ 

5) <in,d = argmin^(Ar*_,^0) 

6) ^ P (mm{T : S^|t ^br} = t-d+l^ 
where ||c|| denotes the Hamming weight of c. 



'while for a given generator matrix, thie parity check matrix is not unique, when Gn,R is block lower, it is easy to see that 
Hn,R can also be chosen to be block lower. 



B. A Sufficient Condition 

The objective is to study the existence of causal Unear codes which under ML decoding guarantee 

Pit < 7/2-/^^ y t, d>do (6) 

where do is a constant independent of d, t. In what follows, we will develop a sufficient condition for a 
Unear code to be anytime reliable in terms of its weight distribution. Suppose the decoding instant is t 
and without loss of generality, assume that the all zero codeword is transmitted, i.e., Cr = for t < t. 
We are interested in the error event where the earliest error in estimating br happens at t = t — d + 1, 
i.e., = for all r < t — 0? + 1 and b^_d_^_^^ 7^ 0. Note that this is equivalent to the ML codeword, c, 
satisfying Cr<ct-d+i = and ct-d+i / 0, and ^ having full rank so that c can be uniquely mapped 
to a transmitted sequence b. Then we have 



PL = P 



is decoded as c 



< ^ P (0 is decoded as c) (7a) 



Now, it is well known (for eg, see [10]) that, under maximum likelihood decoding, P (0 is decoded as c) < 
C}\'^\\, where C, is the Bhattacharya parameter, i.e.. 



00 

C= j \/p{z\X = l)p{z\X = 0)dz 



where, z and X denote the channel output and input respectively. From (7a), it follows that 



If wl^^^^ > and and N^^^ < 2'^'" for some 9 < log2(l/C), then 

Pf^ < r^2~""'^('°S2(i/C)-f) (g) 

where rj = {1 — 2'°S2(i/C)-^)-i. So, an obvious sufficient condition for H„ can be described in terms 
of wl^^^^ and ^ as follows. For some 9 < Iog2(l/C)> we need 

<in,d > arid, Ni^a <2^^ y t, d> do (9a) 

where do is a constant that is independent of d, t. This brings us to the following definition 

Definition 2 (Anytime distance and Anytime reliability): We say that a code B[„ has (a, 9, do)— anytime 



distance, if the following hold 

1) H^^ is full rank for alH > 

2) 'wLm,d ^ ^l,d ^ 2^^^ for all t > and d > do- 

Also, we say that a code H„^i^ is {R, P, do)— anytime reliable if, under ML decoding 

Pt,d < ??2~^"'^, V i > 0, d>do (9b) 
where, do is a constant independent of t,d. 

IV. Linear Anytime Codes - Existence 

We will begin by proving the existence of such codes over a finite time horizon, T, i.e., P^^ < 
r/2~^'^, V t <T, d> do- We will then prove their existence for all time. 

A. Finite Time Horizon 

Over a finite time horizon, T, a causal linear code is represented by a block lower triangular parity 
check matrix H„ r € {0, i}"^x"^. The following Theorem guarantees the existence of a E[„ r such 
that (9) is true for all t < T. 

Theorem 4.1 (Appropriate Weight Distribution): For each time T > 0, rate i? > 0, a < H~^(l — R) 
and 9 > log2(l/(2^~-'^ — 1)), there exists a causal linear code H{n,k,T) that has (q, 0, do)— anytime 
distance, where do is a constant independent of d, t and T. 

H~^{1 — R) is the smaller root of the equation H{x) = 1 — R, where H{.) is the binary entropy function. 
The proof is by induction and is detailed in the Appendix. Theorem 4.1 proves the existence of finite 
dimensional causal linear codes, H„ ^ r> that are anytime rehable for decoding instants upto time T. 
In the following subsection, we demonstrate the existence of infinite dimensional causal linear codes, 
EIn,i?, that are anytime reliable for all decoding instants. We also show that such codes drawn from an 
appropriate ensemble are anytime rehable with a high probability. 



B. Time Invariant Codes 

Consider causal linear codes with the following Toeplitz structure 



Hi 

H2 Hi 

Hf Ht—i . . . Hi 

The superscript TZ in Hj^'^ denotes 'Toeplitz'. is obtained from H„ in (5) by setting Hij = 
for i > j. Due to the Toeplitz structure, we have the following invariance, w^^^^^ = w^^^^ and 
^ = N*' , for all t,i! . The code H!^^ will be referred to as a time-invariant code. The notion of time 
invariance is analogous to the convolutional structure used to show the existence of infinite tree codes in 
[1]. This time invariance allows one to prove that such codes which are anytime reliable are abundant. 

Definition 3 (The ensemble TZp): The ensemble TZp of time-invariant codes, H^^, is obtained as 
follows. Hi is any full rank binary matrix and for r > 2, the entries of Ht are chosen i.i.d according to 
Bemoulh(p), i.e., each entry is 1 with probability p and otherwise. 
For the ensemble TZp, we have the following result 

Theorem 4.2 (Abundance of time-invariant codes): For each i? > 0, a < H~^ [(1 — i?)log2(l/(l — p))\ 
and 9> - logs [(1 - - l] , we have 

P {Ul^R has (a, 0, do) - anytime distance) > 1 - 2-"("'^<') (10) 

We can now use this result to demonstrate an achievable region of rate-exponent pairs for a given channel, 
i.e., the set of rates R and exponents /3 such that one can guarantee {R, j3) anytime reliabihty using hnear 
codes. To determine the values of R that will satisfy (8), note that we need 

log2(l/(2l-^ - 1)) < log2(l/C) ^ R<1- log2(l + C) 

With this observation, we have the following Corollary. 
Corollary 4.3: For any rate R and exponent /3 such that 

^ < 1 - iog°( vll"- P)) "^'^ /3<^-^(l-i^)(log.Q)+log.[(l-;.r(^-«)-l]) (lla) 



nTZ 



if EI^^ is chosen from TZp, then 

P {ml^ji is {R, do) - anytime rehable) > 1 - 2-^^""^"^ 

Note that by choosing p small, we can trade off better rates and exponents with sparser parity check 
matrices. 

C. Tightening the Union Bound 

The region of achievable rate exponent pairs as described in Corollary 4.3 have been obtained by using 
a simple union bound on the error probability. It is well known that for linear block codes, the union 
bound does not give an error exponent for rates close to the chaimel capacity. This appears to be the 
case in Corollary 4.3 as well. For BSC(e) with p = ^, the threshold for rate in Corollary 4.3 becomes 
i? < 1 — 2 log2(-v/e + \/l — e). It turns out that there is a regime of bit flip probabilities, e < e*, where 
this threshold can be improved as follows. 

Theorem 4.4 (Tighter bounds for BSC(e)): Let e* be defined as 

e* = argmin{l - H{2e) < 1 - 21og2(Ve + VT^)} (12) 

e>0 

Consider BSC(e) for e < e*, then for any rate R and exponent /3 such that 

R<1- H{2e), /3<KL - 

if is chosen from TZ i , then 

P {ml% is {R, /3, do) - anytime rehable) > 1 - 2-"("'^°) 

A simple numerical calculation gives e* « 0.0753. Hence for a binary symmetric channel with bit flip 
probability smaller than 0.0753, one can get an anytime exponent for rates closer to the Shaimon capacity 
than that suggested by a simple union bound. In general, given a hnear block code with a specified weight 
distribution, there are numerous results in literature (e.g., see the survey [10]) that sigiuficantly tighten 
the union bound. Some of these techniques will directly yield improved thresholds than those obtained 
in Corollary 4.3. 

D. Stabilizable Region 

Using the thresholds obtained in Corollary 4.3 and Theorem 4.4, we can discuss the range of |A| for 
which the m*'^ moment of xt in (3) can be stabilized over some common channels. Using Lemma 2.1, 
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Fig. 3: Comparing the stabilizable regions of BSC and BEC using linear codes 



an anytime reliable code with rate R and exponent /3 can stabilize the process in (3) for all A such that 

n/3 

logo A < minjnit, — } 

m 

So, a scalar unstable linear process in (3) can be stabilized over a channel with Bhattacharya parameter 
C provided 

/3n 

log2 |A| < log2 \Xmax\ = sup mm{nR, — } (13) 

R<Rc,l3<f5(,n m 

where R^; = 1 - log2(l + C) and P^^r = H-^{1 - R) (\og2 (^j) + loga (2^--^ - l)) are obtained by 
setting p = 2 in (11). For the BSC(e), using Theorem 4.4, one can tighten (13) by replacing i?^ with 
1 - H{e) and with KL {H-^{1 - R)\\ min{e, 1 - e}). For m = 2, the stabilizable region for the 
BEC and BSC is shown in Fig 3 where |Amax|" is plotted against the channel parameter. 

V. Decoding over the BEC 

Owing to the simplicity of the erasure channel, it is possible to come up with an efficient way to 
perform maximum likelihood decoding at each time step. We will show that the average complexity of 
the decoding operation at any time t is constant and that it being larger than KC"^ decays exponentially 
in C. Consider an arbitrary decoding instant t, let c = [cj , . . . jcff^ be the transmitted codeword and 
let z = [zf, . . . , zf]'^ denote the corresponding channel outputs. Recall that denotes the nt x nt 
leading principal minor of ]HI„ Let Ze denote the erasures in z and let He denote the columns of 
that correspond to the positions of the erasures. Also, let z^. denote the unerased entries of z and let 
He denote the columns of excluding He- So, we have the following parity check condition on Ze, 



HeZe = HgZe. Since Ze is known at the decoder, s = HgZe is known. Maximum likelihood decoding 
boils down to solving the linear equation Hf-Zg = s. Due to the lower triangular nature of He, unlike in 
the case of traditional block coding, this equation will typically not have a unique solution, since Hg will 

typically not be full rank. This is alright as we are not interested in decoding the entire z^, correctly, we 
only care about decoding the earher entries accurately. If Zg = [zJi, ^^2]"^' ^^^^ ^e,i corresponds to the 
earlier time instants while Ze,2 corresponds to the latter time instants. The desired reliability requires one 
to recover Ze,i with an exponentially smaller error probability than Ze,2- Since Hg is lower triangular, we 
can write HgZe = s as 
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Ze,l 




Si 


He,21 
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S2 



(14) 



Let H^22 denote the orthogonal complement of i?e,22> ie., H^22He,22 = 0. Then multiplying both sides 
of (14) with diag(/, He,22), we get 



He,ll 




Si 




Ze,l = 




_ Ht,22He,21 _ 




_ H^22S2 _ 



If [Hj^i (i?^22^e,2i)^]^ has full column rank, then Ze,i can be recovered exactly. The decoding 
algorithm now suggests itself, i.e., find the smallest possible He,22 such that [Hj^-^ (iJ^22^e,2i)^]^ 
has full rank and it is outlined in Algorithm 1. 



Algorithm 1 Decoder for the BEC 



1) Suppose, at time t, the earliest uncorrected error is at a delay d. Identify Zg and H^ as defined above. 

2) Starting with d' = l,2,...,d, partition 



[zL zLf and H^ 



He,U 

H 







e,21 



H, 



e,22 



where Ze,2 correspond to the erased positions up to delay d'. 

He,ll 
_ H^22He,21 _ 

4) If so, solve for Ze,i in the system of equations 



3) Check whether the matrix 



has full column rank. 



He,ll 
Ht,22He,21 





Si 


Ze,l = 


_ H^22^2 



5) Increment t = t + l and continue. 
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A. Complexity 

Suppose the earliest uncorrected error is at time t — d+1, then steps 2), 3) and 4) in Algorithm 1 can be 
accomplished by just reducing He into the appropriate row echelon form, which has complexity O(d^). 
The earliest entry in is at time t — d+1 implies that it was not corrected at time t — 1, the probability of 
which is P^_ii_i < r]2~"-^^^~^\ Hence, the average decoding complexity is at most K ^^^Qd^2~"'^^ 
which is bounded and is independent of t. In particular, the probability of the decoding complexity being 
Kd^ is at most ry2~"^'^. The decoder is easy to implement and its performance is simulated in Section 
VI. Note that the encoding complexity per time iteration increases linearly with time. This can also be 
made constant on average if the decoder can send periodic acks back to the encoder with the time index 
of the last correctly decoded source bit. 

VI. Simulations 

Consider stabilizing the scalar unstable process of (3) with A = 2, and wt and vt being uniform on 
[—30, 30] and [—1, 1] respectively, over a binary erasure channel with erasure probability e = 0.3. Also, 
let n = 15, i.e., we get 15 channel uses per time instant. To quantize the measurements, we shall use 
a L-regular lattice quantizer with bin width 6 > V which is depicted in Fig 4 (see for e.g., [5]) If at 
some time t, xt G i—^,^) and the controller is aware of it, then, yt+i € [— — |A|(^, \X\6 + ^^^ ] 
and hence can lie in almost one of ^"'"^^^^''^^'^ possible bins. So, if L > Z+'^^+^lAM^ then the observer 
can encode the bin label of yt+i as bt G {0, l}'^, while the channel coder produces the channel inputs 
ct G {0, 1}". Clearly we need 2'^ > L. So, for 6 = V, L = 35 and hence k = 6, the associated 
code rate R = 6/15 < 1 — log2(l + 0.3) = 0.6215. Using Lemma 2.1, inorder to stabilize \xt\, one 
needs a code with exponent /? > ^ = 0.0667. Using Corollary 4.3, causal linear codes exist for /? < 
(3* = H'^{1 - R) (^log2 (^^^ + log2 (2^-^ ~ ^))- ^ l"^'^'^ calculation shows that for /c = 6,n = 15, 
(3* = = 0.0761 > 0.0667. A time invariant code Mi^^a G TZi was randomly generated and used 



with the decoding algorithm in Section V. Fig 5 shows the plot of a sample path of the above process 
before and after closing the loop, the fact that the plant has been stabihzed is clear. 
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By easing up on the rate R, the control performance of the code ensemble improves. This is demon- 
strated in Fig 6. In the example considered above, one can decrease the rate by increasing 6, e.g., increasing 
6 from V to 4V, k drops from 6 to 4 bits and the resulting rate drops from 0.4 to 0.267. For each value 
of k from 3 to 6, 1000 time invariant codes were generated at random from TZi. Each such code was 

2 

used to control the process above over a time horizon of T = 100. The y— axis denotes sup^^^go \ xt\ 




while the x— axis denotes the proportion of codes for which sup^^^QQ \xt\ is below a prescribed value. 
For example, with k = 6,n = 15, sup^^ioo l^^*! was less than 200 for 50% of the codes while with 
/c = 3, n = 15, this fraction increases to more than 95%. The y— axis has been capped at 1000 for clarity. 
This shows that one can trade-off utilization of communication resources and control performance. A 
rate R = 0.267 is much smaller than the capacity 1 — log2(l + 0.3) = 0.6215 but allows codes with 
much better control performance. 

VII. Conclusion 

We prove the existence of linear anytime codes with high probability. Our analysis considered binary 
alphabet, but is easily extendable. This is a significant step, since prior work only demonstrated the 
existence of such codes. For the BEC, we also propose an efficient decoding algorithm with constant 
average complexity per iteration, and for which the probability of having a complexity of KC^ at some 
given iteration decays exponentially in C. Simulations validate the efficacy of the method. Constructing 
code families with efficient decoding for other channels, such as the BSC or AWGN remains an interesting 
open problem. 
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Appendix 

Lemma A.1: Let V = {0, 1}"* and define a probability function over V such that, for each v ^V, 

P{v) =]9'l^'l(l — pY'--\\'"\\ , If [/ is a £— dimensional subspace of V, then 



P{U) < < 



(1 -p)"^-^ if p < 1/2 

pm-e jf p>l/2 

The proof is easy. 

Remark 1 (Rank of a block lower traingular matrix): Let A be the following block lower triangular 
matrix 



A = 



An ... 

A21 A22 



An A. 



t2 



At 



If each of {^jj}j=i has full rank, then A also has full rank. 

Proof of Theorem 4.2: Consider an arbitrary decoding instant, t. Since u;^; 
d ^' * ' ^i^^ '^^P these superscripts and write 



t 



V 

min.c! 



W 



t 

min,(i 



^^min,rf and Nl , 



and iV^,rf = 
= Let 



c = [c{ , . . . ,cl\ , where Cj G {0, 1}'*, be a fixed binary word such that CT<t-d+i = and ct-d+i 7^ 0. 



Also, let H„ be drawn from the ensemble TZp and let ^ denote the nt x nt principal minor of H„ 7^. 
We examine the probabihty that c is a codeword of ^, i.e., P ^H^ = . Now, since c^<t_d+i = 0' 



i?c = is equivalent to 



Hi 







Ct-d+1 







H2 


Hi ... 




Ct-d+2 







Hd 


^^d-l ... Hi 




Ct 








(16) 



Note that (16) can be equivalently written as follows 



Ct-d+1 







hi 







Ct-d+2 


Ct-d+1 




h2 







Ct 


Ct-1 ■ ■ ■ Ct-d+1 




. . 









4c 









(17) 



where hi = yQc{Hj), i.e., is a nn x 1 column obtained by stacking the columns of Hf one below the 
other, and Cj G {0, 1}"^^"" is obtained from Cj as follows. 



Ci 







Since ct-d+i 0' Q-d+i has full rank n and consequently C has full rank dn. So, /'(H^ = 0) = 
P{Ch = 0). This is the probability that h hes in a dnn — dfi dimensional null space of C. This further 
imphes that h' = [/t^, . . . , h'^]'^ hes in a (d — l)nn — (d — l)n dimensional subspace. Note that h' is an 
(d — l)nn dimensional Bemoulh(p) vector and using Lemma A.l, we have 



nd 
w' 



P(H^,^c = 0)<(l-p)^('^-i) 
P{w^,^^,<and)<{l-pr^^-'^ E 

w'<and 

< (1 _ jr,)"('^-l)2"'^^W 

^ ^^-ndi(l-R)log^(l/(l-p))-H(a)) 



(18) 



(19) 



where 77 = (1 — p) Similarly, 



< ??2 



< ??2- 



w 



-nd{ew/nd-H{w/nd)+{l-R)\og2{l/{l-p))) 



(20) 



For convenience, define 

ii = (l-/;)log2(l/(l-p))-H(a) 
We need to choose 6 such that S2,w > > for all a < ^ < 1. Now, define 



(21) 



Then for each 9 > 9*, there is a 5 > such that S2,w > <^ for all and < w < nd. A simple calculation 
gives 6* = log2 For such a choice of > 0*, continuing from (20), we have 



P ( 3and < u; < 3 N^d> 2^"" I < rzd2-"'^'^ 



(22) 



for some (5' > 0. For some fixed do large enough, applying a union bound over d > do to (19) and (22), 
we get 



p(3d>do 3 w^i^^a < and or N^^^ > 2^^^) < 2-^^""^"^ 



(23) 



Recall that for the above argument to be complete, we also require ^ to have full rank for each t. 
By Remark 1, this is guaranteed if Hi has full rank. So, ■ 

Proof of Theorem 4.1: The proof is by induction. Suppose Mn^R^x-i has (a, 9, do)— anytime distance. 
Construct B[„ ^ t as follows. 



H 



11 







H- 



21 



lln,R,r-l 



where Hn is chosen to be a full rank matrix and the entries of Hji G {0, l}'*^", j > 2, are drawn 
according to i.i.d Bemoulh(^). We will show that if H„ ijT-i has (a, ^, do)— anytime distance, then 
H„,ij,T will is also have (a, 6, do)— anytime distance with a probability 1 — 2~^("'*''). Note that the prob- 
ability is over the choice of {Hji}J^^^. Let {^Wj^jn ^, d}d>d„,t<T be the weight distribution parameters 
associated to M^^r^t- Since M.n,R,T-i has (a, ^, do)— anytime distance, we have the following 

^min d — ctnd, \/ do < d <t — \, t > do + I 
^L,d < 2^"'. V u; > and, do<d<t-l, t > do + I 

Towards proving that Mn^R^r has (a, 0, do)— anytime distance, it remains to show the following holds 
with a positive probability. 

For t > do, u;min,t > ant, A^4,t <^^'^, V u; > ant (24) 

Let c G {0, 1}"* such that CT^t~d+i = and ct-d+i 0' then it is easy to see that P ^H^ ij — ^) — 
2-n(<i-i) jY^Q j.gg|^ i^jjg analysis follows exactly along the hnes of the proof of Theorem 4.2 starting 
from (18) with p = |. This gives the following result 

P (Mn,R,T is bad|E[„^R_T-i is good) = 

P ({<in,d, K,d} do not satisfy (24)) < 1 - 2-^("''<') 

In particular, there exists a choice of such that Mn^R^r has (a, ^, do)— anytime distance, 

whenever H^ r t-i has (a, 0, do)— anytime distance. For the inductive argument to be complete, one 
needs to prove that there exists a Mn^R^d^ that has (a, ^, do)— anytime distance. This is already covered 
in the proof of the above inductive step. ■ 
Proof of Theorem 4.4: The result can be obtained by a slight modification of the proof of Theorem 
4.2. Consider an arbitrary decoding instant t, if the all zero codeword is transmitted is the transmitted 
codeword, z = [zf, . . . , zJY is the channel output and v = [v\,..., vJY is the bit flip pattern induced 



by the channel BSC(e), e < 1/2, then z = v and for any c G Ct,d, we have 

Pt,d ^ P \ U *^ decoded as c, ||i;t_d+i:t|| < nd(e + S) 
+ P(||i;t_d+i:t|| >nd{e + 6)) 

< Yl iV^,dC + 2""'^-^^^'+^""'^ (25) 

Note that if lo^in^d > 2nd(e + (^), then we get P^^^ < 2-"'^^^(«+'5|l«). Since we know from Theorem 4.2 
that u^ujin ^ can scale as and for a upto H~^{1 — R), for any choice of R such that H~^{1 — R) > 2e, 
there exists ad > small enough such that 2e + 2S < a < H~^{1 — R). Fix e < 0.25, then for each rate 
R < 1 — H(2e), one can achieve an exponent, /3, upto P < KL {^H~^{1 — i?)||e). It is easy to see that 
this marks an improvement over Corollary 4.3 only for e such that 1 — H{2e) > 1 — 2 log2 {s/e + ^/T^^) . 
The result is now immediate. As an aside, a quick numerical calculation gives e* 0.0753. ■ 



